<!DOCTYPE html>


<html lang="zh-CN">


<head>
  <meta charset="utf-8" />
    
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1" />
  <title>
    Kubernetes 临时存储需要限制吗.md |  
  </title>
  <meta name="generator" content="hexo-theme-ayer">
  
  <link rel="shortcut icon" href="/favicon.ico" />
  
  
<link rel="stylesheet" href="/dist/main.css">

  
<link rel="stylesheet" href="https://cdn.jsdelivr.net/gh/Shen-Yu/cdn/css/remixicon.min.css">

  
<link rel="stylesheet" href="/css/custom.css">

  
  
<script src="https://cdn.jsdelivr.net/npm/pace-js@1.0.2/pace.min.js"></script>

  
  

  

</head>

</html>

<body>
  <div id="app">
    
      
    <main class="content on">
      <section class="outer">
  <article
  id="post-k8s/Kubernetes 临时存储需要限制吗"
  class="article article-type-post"
  itemscope
  itemprop="blogPost"
  data-scroll-reveal
>
  <div class="article-inner">
    
    <header class="article-header">
       
<h1 class="article-title sea-center" style="border-left:0" itemprop="name">
  Kubernetes 临时存储需要限制吗.md
</h1>
 

    </header>
     
    <div class="article-meta">
      <a href="/2020/11/11/k8s/Kubernetes%20%E4%B8%B4%E6%97%B6%E5%AD%98%E5%82%A8%E9%9C%80%E8%A6%81%E9%99%90%E5%88%B6%E5%90%97/" class="article-date">
  <time datetime="2020-11-10T16:00:00.000Z" itemprop="datePublished">2020-11-11</time>
</a> 
  <div class="article-category">
    <a class="article-category-link" href="/categories/k8s/">k8s</a>
  </div>
  
<div class="word_count">
    <span class="post-time">
        <span class="post-meta-item-icon">
            <i class="ri-quill-pen-line"></i>
            <span class="post-meta-item-text"> 字数统计:</span>
            <span class="post-count">1.3k</span>
        </span>
    </span>

    <span class="post-time">
        &nbsp; | &nbsp;
        <span class="post-meta-item-icon">
            <i class="ri-book-open-line"></i>
            <span class="post-meta-item-text"> 阅读时长≈</span>
            <span class="post-count">4 分钟</span>
        </span>
    </span>
</div>
 
    </div>
      
    <div class="tocbot"></div>




  
    <div class="article-entry" itemprop="articleBody">
       
  <h1 id="Kubernetes-临时存储需要限制吗"><a href="#Kubernetes-临时存储需要限制吗" class="headerlink" title="Kubernetes 临时存储需要限制吗"></a>Kubernetes 临时存储需要限制吗</h1><h2 id="临时存储简介"><a href="#临时存储简介" class="headerlink" title="临时存储简介"></a>临时存储简介</h2><p><code>Node节点</code>通常还可以具有本地的临时性存储，由本地挂载的<code>可写入设备</code>或者 <code>RAM</code>来提供支持。<code>临时（Ephemeral）</code> 意味着对所存储的数据不提供长期可用性的保证。</p>
<p>Pods 通常可以使用临时性本地存储来实现缓冲区、保存日志等功能。kubelet 可以为使用本地临时存储的 Pods 提供这种存储空间，允许后者使用 <code>emptyDir</code> 类型的卷将其挂载到容器中。</p>
<p>kubelet 也使用此类存储来保存<code>节点层面的容器日志</code>， <code>容器镜像文件</code>、<code>以及运行中容器的可写入层</code>。</p>
<h2 id="临时存储有哪些"><a href="#临时存储有哪些" class="headerlink" title="临时存储有哪些"></a>临时存储有哪些</h2><ul>
<li>本地临时存储（local ephemeral storage）</li>
<li>emptyDir</li>
</ul>
<p><code>本地临时存储（local ephemeral storage）</code>：Kubernetes在1.8的版本中引入了一种类似于CPU，内存的新的资源模式：ephemeral-storage，并且在1.10版本kubelet中默认打开这个特性。ephemeral-storage是为管理和调度Kubernetes中运行的应用短暂存储。</p>
<p><code>emptyDir</code>：emptyDir 类型Volume在Pod分配到Node上时被创建，Kubernetes会在Node节点上自动分配一个目录，因此无需指定宿主机Node上对应的目录文件。这个目录初始内容为空，当Pod从Node上移除时，emptyDir中的数据会被永久删除。</p>
<blockquote>
<p>注释：容器的 <code>crashing</code> 事件并不会导致 <code>emptyDir</code> 中的数据被删除。</p>
</blockquote>
<h2 id="临时存储默认存储在哪个位置"><a href="#临时存储默认存储在哪个位置" class="headerlink" title="临时存储默认存储在哪个位置?"></a>临时存储默认存储在哪个位置?</h2><p>在每个 Kubernetes <code>Node节点</code> 上，kubelet 默认根目录是 <code>/var/lib/kubelet</code> 和 日志目录 <code>/var/log</code> 保存在节点的系统分区上，这个分区同时也会被Pod的 <code>EmptyDir</code> 类型的<code>volume</code>、<code>容器日志</code>、<code>镜像层</code>、<code>容器的可写层所占用</code>。<code>ephemeral-storage</code> 便是对系统分区进行管理。</p>
<h2 id="临时存储需要限制吗？"><a href="#临时存储需要限制吗？" class="headerlink" title="临时存储需要限制吗？"></a>临时存储需要限制吗？</h2><p>答案是 <code>需要限制</code>，从上文了解到，临时存储默认根目录是在 <code>/var/lib/kubelet</code>中，<code>/var</code> 一般情况都是在系统根分区中，并且根分区磁盘一般不会很大（阿里云ECS系统盘默认为 40G），这就必须限制，为系统预留足够的磁盘空间来支持正常运行。上文也说到，临时存储也可以使用 RAM，那就更应该限制，内存是一种非常有限的资源。</p>
<h2 id="Node节点设置临时存储使用大小"><a href="#Node节点设置临时存储使用大小" class="headerlink" title="Node节点设置临时存储使用大小"></a>Node节点设置临时存储使用大小</h2><p>Node节点上的 <code>kubelet</code> 组件启动时，kubelet会统计当前节点默认 <code>/var/lib/kubelet</code> 所在的分区可分配的磁盘资源，或者你可以覆盖节点上kubelet的配置来自定义可分配的资源。创建Pod时会根据存储需求调度到满足存储的节点，Pod使用超过限制的存储时会对其做<code>驱逐</code>处理来保证不会耗尽节点上的磁盘空间。</p>
<blockquote>
<p>注意：如果运行时指定了别的独立的分区，比如修改了docker的镜像层和容器可写层的存储位置(默认是/var/lib/docker)所在的分区，将不再将其计入 <code>ephemeral-storage</code> 的消耗。</p>
</blockquote>
<p>kubelet 如下配置，限制Node节点上临时存储能使用多大磁盘空间</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># Node 资源保留</span><br><span class="line">  nodefs.available: 10% # 给 &#x2F;var&#x2F;lib&#x2F;kubelet 所在分区保留 10% 磁盘空间</span><br><span class="line">  nodefs.inodesFree: 5% # 给 &#x2F;var&#x2F;lib&#x2F;kubelet 所在分区保留 5% inodes</span><br></pre></td></tr></table></figure>

<h2 id="临时存储限制使用举例"><a href="#临时存储限制使用举例" class="headerlink" title="临时存储限制使用举例"></a>临时存储限制使用举例</h2><h3 id="限制磁盘本地临时存储"><a href="#限制磁盘本地临时存储" class="headerlink" title="限制磁盘本地临时存储"></a>限制磁盘本地临时存储</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">apiVersion: v1</span><br><span class="line">kind: Pod</span><br><span class="line">metadata:</span><br><span class="line">  name: test-storage</span><br><span class="line">  labels:</span><br><span class="line">    app: test-storage</span><br><span class="line">spec:</span><br><span class="line">  containers:</span><br><span class="line">  - name: busybox</span><br><span class="line">    image:  busybox</span><br><span class="line">    command: [&quot;sh&quot;, &quot;-c&quot;, &quot;while true; do dd if&#x3D;&#x2F;dev&#x2F;zero of&#x3D;$(date &#39;+%s&#39;).out count&#x3D;1 bs&#x3D;30MB; sleep 1; done&quot;] # 使用dd命令持续往容器写数据</span><br><span class="line">    resources:</span><br><span class="line">      limits:</span><br><span class="line">        ephemeral-storage: 300Mi #定义存储的限制为300Mi</span><br><span class="line">      requests:</span><br><span class="line">        ephemeral-storage: 300Mi</span><br></pre></td></tr></table></figure>

<p>容器使用磁盘超过 300Mi，被 kubelet 驱逐。具体请见下图</p>
<p><img src="http://iubest.gitee.io/pic/640-1601018227635.png" alt="img"></p>
<h3 id="限制内存临时存储"><a href="#限制内存临时存储" class="headerlink" title="限制内存临时存储"></a>限制内存临时存储</h3><p><code>emptyDir</code> 也是一种临时存储，因此也需要限制使用。</p>
<p>在Pod级别检查临时存储使用量时，也会将 <code>emptyDir</code> 的使用量计算在内，因此如果对 emptyDir 使用过量后，也会导致该Pod被 kubelet <code>Evict</code>。</p>
<p>另外，emptyDir本身也可以设置容量上限。指定 emptyDir 使用内存作为存储介质，这样用户可以获得极好的读写性能，但是由于内存比较珍贵，只提供了 <code>128Mi</code> 的空间，当用户在 <code>/cache-data</code> 目录下使用超过64Mi后，该Pod会被 kubelet 驱逐。</p>
<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">  volumeMounts:</span><br><span class="line">  - mountPath: &#x2F;cache-data</span><br><span class="line">    name: cache-data-volume</span><br><span class="line">volumes:</span><br><span class="line">- emptyDir:</span><br><span class="line">    medium: Memory</span><br><span class="line">    sizeLimit: 128Mi</span><br><span class="line">  name: cache-data-volume</span><br></pre></td></tr></table></figure>

<h2 id="参考链接"><a href="#参考链接" class="headerlink" title="参考链接"></a>参考链接</h2><ul>
<li><a target="_blank" rel="noopener" href="https://kubernetes.io/zh/docs/concepts/configuration/manage-resources-containers/">https://kubernetes.io/zh/docs/concepts/configuration/manage-resources-containers/</a></li>
<li><a target="_blank" rel="noopener" href="https://developer.aliyun.com/article/594066">https://developer.aliyun.com/article/594066</a></li>
<li><a target="_blank" rel="noopener" href="https://ieevee.com/tech/2019/05/23/ephemeral-storage.html">https://ieevee.com/tech/2019/05/23/ephemeral-storage.html</a></li>
</ul>
<h2 id="热门文章推荐"><a href="#热门文章推荐" class="headerlink" title="热门文章推荐"></a>热门文章推荐</h2><ul>
<li><a target="_blank" rel="noopener" href="http://mp.weixin.qq.com/s?__biz=MzA4MzIwNTc4NQ==&mid=2247486468&idx=1&sn=b43b42bffea97cbe0e247d109fc8a60e&chksm=9ffb47f2a88ccee4e8b59958d32d233622c1d938c75397c76aae3b97513182e6d081954ca39e&scene=21#wechat_redirect">分享阿里巴巴云原生技术与实践 - KubeCon 2020 经典演讲集锦</a></li>
<li><a target="_blank" rel="noopener" href="http://mp.weixin.qq.com/s?__biz=MzA4MzIwNTc4NQ==&mid=2247486422&idx=1&sn=48473bcf5c7f9451569137f5bbdf95a3&chksm=9ffb4020a88cc93603d5b2094a9b0c198e073bfbaf4cc4b0f82ef038a488371c3e853773894a&scene=21#wechat_redirect">Kubernetes v1.19.0 正式发布！</a></li>
<li><a target="_blank" rel="noopener" href="http://mp.weixin.qq.com/s?__biz=MzA4MzIwNTc4NQ==&mid=2247486324&idx=1&sn=97fb1ca3b706056595754844fe6db5de&chksm=9ffb4082a88cc99487ed614c68afbf7549006994e5ecfacf7774aeaca9e68a4a436d54542e39&scene=21#wechat_redirect">IT运维面试问题总结-简述Etcd、Kubernetes、Lvs、HAProxy等</a></li>
<li><a target="_blank" rel="noopener" href="http://mp.weixin.qq.com/s?__biz=MzA4MzIwNTc4NQ==&mid=2247485830&idx=1&sn=8b0031a60fdbc0b080d8b4967f03b859&chksm=9ffb4270a88ccb66c9e0002d7162254c1ca392d9f6da41b5b8bfa046fd1e07dc7ed3d84856a1&scene=21#wechat_redirect">Kubernetes 升级填坑指南（一）</a></li>
<li><a target="_blank" rel="noopener" href="http://mp.weixin.qq.com/s?__biz=MzA4MzIwNTc4NQ==&mid=2247485859&idx=1&sn=c1355c41c67dcb28453158a38bf15f81&chksm=9ffb4255a88ccb430a26a3bf988a55302adff24c303fd85285372d0ab6e8fb94ae26ba3d485d&scene=21#wechat_redirect">Kubernetes v1.15.3 升级到 v1.18.5 心得</a></li>
<li><a target="_blank" rel="noopener" href="http://mp.weixin.qq.com/s?__biz=MzA4MzIwNTc4NQ==&mid=2247485814&idx=1&sn=fc3545f9d8fa7d20274195dd8409d4dd&chksm=9ffb4280a88ccb9617ebd256a9b96a062a6dbf2e23ebd9e507599c9a574b64b76f36029f9a25&scene=21#wechat_redirect">根据 PID 获取 K8S Pod名称 - 反之 POD名称 获取 PID</a></li>
</ul>
 
      <!-- reward -->
      
      <div id="reword-out">
        <div id="reward-btn">
          打赏
        </div>
      </div>
      
    </div>
    

    <!-- copyright -->
    
    <div class="declare">
      <ul class="post-copyright">
        <li>
          <i class="ri-copyright-line"></i>
          <strong>版权声明： </strong>
          
          本博客所有文章除特别声明外，著作权归作者所有。转载请注明出处！
          
        </li>
      </ul>
    </div>
    
    <footer class="article-footer">
       
<div class="share-btn">
      <span class="share-sns share-outer">
        <i class="ri-share-forward-line"></i>
        分享
      </span>
      <div class="share-wrap">
        <i class="arrow"></i>
        <div class="share-icons">
          
          <a class="weibo share-sns" href="javascript:;" data-type="weibo">
            <i class="ri-weibo-fill"></i>
          </a>
          <a class="weixin share-sns wxFab" href="javascript:;" data-type="weixin">
            <i class="ri-wechat-fill"></i>
          </a>
          <a class="qq share-sns" href="javascript:;" data-type="qq">
            <i class="ri-qq-fill"></i>
          </a>
          <a class="douban share-sns" href="javascript:;" data-type="douban">
            <i class="ri-douban-line"></i>
          </a>
          <!-- <a class="qzone share-sns" href="javascript:;" data-type="qzone">
            <i class="icon icon-qzone"></i>
          </a> -->
          
          <a class="facebook share-sns" href="javascript:;" data-type="facebook">
            <i class="ri-facebook-circle-fill"></i>
          </a>
          <a class="twitter share-sns" href="javascript:;" data-type="twitter">
            <i class="ri-twitter-fill"></i>
          </a>
          <a class="google share-sns" href="javascript:;" data-type="google">
            <i class="ri-google-fill"></i>
          </a>
        </div>
      </div>
</div>

<div class="wx-share-modal">
    <a class="modal-close" href="javascript:;"><i class="ri-close-circle-line"></i></a>
    <p>扫一扫，分享到微信</p>
    <div class="wx-qrcode">
      <img src="//api.qrserver.com/v1/create-qr-code/?size=150x150&data=http://example.com/2020/11/11/k8s/Kubernetes%20%E4%B8%B4%E6%97%B6%E5%AD%98%E5%82%A8%E9%9C%80%E8%A6%81%E9%99%90%E5%88%B6%E5%90%97/" alt="微信分享二维码">
    </div>
</div>

<div id="share-mask"></div>  
  <ul class="article-tag-list" itemprop="keywords"><li class="article-tag-list-item"><a class="article-tag-list-link" href="/tags/k8s/" rel="tag">k8s</a></li></ul>

    </footer>
  </div>

   
  <nav class="article-nav">
    
      <a href="/2020/11/11/k8s/Kubernetes%20%E4%B8%8A%E5%AF%B9%E5%BA%94%E7%94%A8%E7%A8%8B%E5%BA%8F%E8%BF%9B%E8%A1%8C%E6%95%85%E9%9A%9C%E6%8E%92%E9%99%A4%E7%9A%84%206%20%E4%B8%AA%E6%8A%80%E5%B7%A7/" class="article-nav-link">
        <strong class="article-nav-caption">上一篇</strong>
        <div class="article-nav-title">
          
            Kubernetes 上对应用程序进行故障排除的 6 个技巧.md
          
        </div>
      </a>
    
    
      <a href="/2020/11/11/k8s/Kubernetes%20%E6%95%85%E9%9A%9C%E8%A7%A3%E5%86%B3%E5%BF%83%E5%BE%97/" class="article-nav-link">
        <strong class="article-nav-caption">下一篇</strong>
        <div class="article-nav-title">Kubernetes 故障解决心得.md</div>
      </a>
    
  </nav>

   
<!-- valine评论 -->
<div id="vcomments-box">
  <div id="vcomments"></div>
</div>
<script src="//cdn1.lncld.net/static/js/3.0.4/av-min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/valine@1.4.14/dist/Valine.min.js"></script>
<script>
  new Valine({
    el: "#vcomments",
    app_id: "",
    app_key: "",
    path: window.location.pathname,
    avatar: "monsterid",
    placeholder: "给我的文章加点评论吧~",
    recordIP: true,
  });
  const infoEle = document.querySelector("#vcomments .info");
  if (infoEle && infoEle.childNodes && infoEle.childNodes.length > 0) {
    infoEle.childNodes.forEach(function (item) {
      item.parentNode.removeChild(item);
    });
  }
</script>
<style>
  #vcomments-box {
    padding: 5px 30px;
  }

  @media screen and (max-width: 800px) {
    #vcomments-box {
      padding: 5px 0px;
    }
  }

  #vcomments-box #vcomments {
    background-color: #fff;
  }

  .v .vlist .vcard .vh {
    padding-right: 20px;
  }

  .v .vlist .vcard {
    padding-left: 10px;
  }
</style>

 
     
</article>

</section>
      <footer class="footer">
  <div class="outer">
    <ul>
      <li>
        Copyrights &copy;
        2015-2020
        <i class="ri-heart-fill heart_icon"></i> TzWind
      </li>
    </ul>
    <ul>
      <li>
        
        
        
        由 <a href="https://hexo.io" target="_blank">Hexo</a> 强力驱动
        <span class="division">|</span>
        主题 - <a href="https://github.com/Shen-Yu/hexo-theme-ayer" target="_blank">Ayer</a>
        
      </li>
    </ul>
    <ul>
      <li>
        
        
        <span>
  <span><i class="ri-user-3-fill"></i>访问人数:<span id="busuanzi_value_site_uv"></span></s>
  <span class="division">|</span>
  <span><i class="ri-eye-fill"></i>浏览次数:<span id="busuanzi_value_page_pv"></span></span>
</span>
        
      </li>
    </ul>
    <ul>
      
    </ul>
    <ul>
      
    </ul>
    <ul>
      <li>
        <!-- cnzz统计 -->
        
        <script type="text/javascript" src='https://s9.cnzz.com/z_stat.php?id=1278069914&amp;web_id=1278069914'></script>
        
      </li>
    </ul>
  </div>
</footer>
      <div class="float_btns">
        <div class="totop" id="totop">
  <i class="ri-arrow-up-line"></i>
</div>

<div class="todark" id="todark">
  <i class="ri-moon-line"></i>
</div>

      </div>
    </main>
    <aside class="sidebar on">
      <button class="navbar-toggle"></button>
<nav class="navbar">
  
  <div class="logo">
    <a href="/"><img src="/images/ayer-side.svg" alt="Hexo"></a>
  </div>
  
  <ul class="nav nav-main">
    
    <li class="nav-item">
      <a class="nav-item-link" href="/">主页</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/archives">归档</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/categories">分类</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/tags">标签</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" target="_blank" rel="noopener" href="http://www.baidu.com">百度</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/friends">友链</a>
    </li>
    
    <li class="nav-item">
      <a class="nav-item-link" href="/2019/about">关于我</a>
    </li>
    
  </ul>
</nav>
<nav class="navbar navbar-bottom">
  <ul class="nav">
    <li class="nav-item">
      
      <a class="nav-item-link nav-item-search"  title="搜索">
        <i class="ri-search-line"></i>
      </a>
      
      
      <a class="nav-item-link" target="_blank" href="/atom.xml" title="RSS Feed">
        <i class="ri-rss-line"></i>
      </a>
      
    </li>
  </ul>
</nav>
<div class="search-form-wrap">
  <div class="local-search local-search-plugin">
  <input type="search" id="local-search-input" class="local-search-input" placeholder="Search...">
  <div id="local-search-result" class="local-search-result"></div>
</div>
</div>
    </aside>
    <script>
      if (window.matchMedia("(max-width: 768px)").matches) {
        document.querySelector('.content').classList.remove('on');
        document.querySelector('.sidebar').classList.remove('on');
      }
    </script>
    <div id="mask"></div>

<!-- #reward -->
<div id="reward">
  <span class="close"><i class="ri-close-line"></i></span>
  <p class="reward-p"><i class="ri-cup-line"></i>请我喝杯咖啡吧~</p>
  <div class="reward-box">
    
    
  </div>
</div>
    
<script src="/js/jquery-2.0.3.min.js"></script>


<script src="/js/lazyload.min.js"></script>

<!-- Tocbot -->


<script src="/js/tocbot.min.js"></script>

<script>
  tocbot.init({
    tocSelector: '.tocbot',
    contentSelector: '.article-entry',
    headingSelector: 'h1, h2, h3, h4, h5, h6',
    hasInnerContainers: true,
    scrollSmooth: true,
    scrollContainer: 'main',
    positionFixedSelector: '.tocbot',
    positionFixedClass: 'is-position-fixed',
    fixedSidebarOffset: 'auto'
  });
</script>

<script src="https://cdn.jsdelivr.net/npm/jquery-modal@0.9.2/jquery.modal.min.js"></script>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/jquery-modal@0.9.2/jquery.modal.min.css">
<script src="https://cdn.jsdelivr.net/npm/justifiedGallery@3.7.0/dist/js/jquery.justifiedGallery.min.js"></script>

<script src="/dist/main.js"></script>

<!-- ImageViewer -->

<!-- Root element of PhotoSwipe. Must have class pswp. -->
<div class="pswp" tabindex="-1" role="dialog" aria-hidden="true">

    <!-- Background of PhotoSwipe. 
         It's a separate element as animating opacity is faster than rgba(). -->
    <div class="pswp__bg"></div>

    <!-- Slides wrapper with overflow:hidden. -->
    <div class="pswp__scroll-wrap">

        <!-- Container that holds slides. 
            PhotoSwipe keeps only 3 of them in the DOM to save memory.
            Don't modify these 3 pswp__item elements, data is added later on. -->
        <div class="pswp__container">
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
            <div class="pswp__item"></div>
        </div>

        <!-- Default (PhotoSwipeUI_Default) interface on top of sliding area. Can be changed. -->
        <div class="pswp__ui pswp__ui--hidden">

            <div class="pswp__top-bar">

                <!--  Controls are self-explanatory. Order can be changed. -->

                <div class="pswp__counter"></div>

                <button class="pswp__button pswp__button--close" title="Close (Esc)"></button>

                <button class="pswp__button pswp__button--share" style="display:none" title="Share"></button>

                <button class="pswp__button pswp__button--fs" title="Toggle fullscreen"></button>

                <button class="pswp__button pswp__button--zoom" title="Zoom in/out"></button>

                <!-- Preloader demo http://codepen.io/dimsemenov/pen/yyBWoR -->
                <!-- element will get class pswp__preloader--active when preloader is running -->
                <div class="pswp__preloader">
                    <div class="pswp__preloader__icn">
                        <div class="pswp__preloader__cut">
                            <div class="pswp__preloader__donut"></div>
                        </div>
                    </div>
                </div>
            </div>

            <div class="pswp__share-modal pswp__share-modal--hidden pswp__single-tap">
                <div class="pswp__share-tooltip"></div>
            </div>

            <button class="pswp__button pswp__button--arrow--left" title="Previous (arrow left)">
            </button>

            <button class="pswp__button pswp__button--arrow--right" title="Next (arrow right)">
            </button>

            <div class="pswp__caption">
                <div class="pswp__caption__center"></div>
            </div>

        </div>

    </div>

</div>

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.css">
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/default-skin/default-skin.min.css">
<script src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/photoswipe@4.1.3/dist/photoswipe-ui-default.min.js"></script>

<script>
    function viewer_init() {
        let pswpElement = document.querySelectorAll('.pswp')[0];
        let $imgArr = document.querySelectorAll(('.article-entry img:not(.reward-img)'))

        $imgArr.forEach(($em, i) => {
            $em.onclick = () => {
                // slider展开状态
                // todo: 这样不好，后面改成状态
                if (document.querySelector('.left-col.show')) return
                let items = []
                $imgArr.forEach(($em2, i2) => {
                    let img = $em2.getAttribute('data-idx', i2)
                    let src = $em2.getAttribute('data-target') || $em2.getAttribute('src')
                    let title = $em2.getAttribute('alt')
                    // 获得原图尺寸
                    const image = new Image()
                    image.src = src
                    items.push({
                        src: src,
                        w: image.width || $em2.width,
                        h: image.height || $em2.height,
                        title: title
                    })
                })
                var gallery = new PhotoSwipe(pswpElement, PhotoSwipeUI_Default, items, {
                    index: parseInt(i)
                });
                gallery.init()
            }
        })
    }
    viewer_init()
</script>

<!-- MathJax -->

<!-- Katex -->

<!-- busuanzi  -->


<script src="/js/busuanzi-2.3.pure.min.js"></script>


<!-- ClickLove -->

<!-- ClickBoom1 -->

<!-- ClickBoom2 -->

<!-- CodeCopy -->


<link rel="stylesheet" href="/css/clipboard.css">

<script src="https://cdn.jsdelivr.net/npm/clipboard@2/dist/clipboard.min.js"></script>
<script>
  function wait(callback, seconds) {
    var timelag = null;
    timelag = window.setTimeout(callback, seconds);
  }
  !function (e, t, a) {
    var initCopyCode = function(){
      var copyHtml = '';
      copyHtml += '<button class="btn-copy" data-clipboard-snippet="">';
      copyHtml += '<i class="ri-file-copy-2-line"></i><span>COPY</span>';
      copyHtml += '</button>';
      $(".highlight .code pre").before(copyHtml);
      $(".article pre code").before(copyHtml);
      var clipboard = new ClipboardJS('.btn-copy', {
        target: function(trigger) {
          return trigger.nextElementSibling;
        }
      });
      clipboard.on('success', function(e) {
        let $btn = $(e.trigger);
        $btn.addClass('copied');
        let $icon = $($btn.find('i'));
        $icon.removeClass('ri-file-copy-2-line');
        $icon.addClass('ri-checkbox-circle-line');
        let $span = $($btn.find('span'));
        $span[0].innerText = 'COPIED';
        
        wait(function () { // 等待两秒钟后恢复
          $icon.removeClass('ri-checkbox-circle-line');
          $icon.addClass('ri-file-copy-2-line');
          $span[0].innerText = 'COPY';
        }, 2000);
      });
      clipboard.on('error', function(e) {
        e.clearSelection();
        let $btn = $(e.trigger);
        $btn.addClass('copy-failed');
        let $icon = $($btn.find('i'));
        $icon.removeClass('ri-file-copy-2-line');
        $icon.addClass('ri-time-line');
        let $span = $($btn.find('span'));
        $span[0].innerText = 'COPY FAILED';
        
        wait(function () { // 等待两秒钟后恢复
          $icon.removeClass('ri-time-line');
          $icon.addClass('ri-file-copy-2-line');
          $span[0].innerText = 'COPY';
        }, 2000);
      });
    }
    initCopyCode();
  }(window, document);
</script>


<!-- CanvasBackground -->


    
  </div>
</body>

</html>